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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
Before the Board of Patent Appeals and Interferences 

In re patent application of 

Hideki Sawada Confirmation No. 1 193 

Serial No. 09/837,190 Group Art Unit 263-1 

Filed April 19, 2001 Examiner Jamie J. Vent 
For RECORDING/REPRODUCING SYSTEM 

Commissioner for Patents 
PO Box 1450 

Alexandria, Virginia 22313-1450 

APPELLANT'S BRIEF UNDER 37 C.F.R. §41.37 

This brief is in furtherance of the Notice of Appeal filed on November 13, 
2006. This appeal is taken under 37 C.F.R. §41.31 (a)(1) from the rejection in the 
Office Action mailed July 14, 2006, which Office Action was as a resuk of the Notice 
of Panel Decision from Pre-Appeal Brief Review mailed April 25, 2006, reopening 
prosecution. The Office Action mailed July 14, 2006, while not a final rejection, is 
substantially a repetition of the final rejection mailed January 11, 2006, but with the 
addition of yet another reference. 

This brief contains these items under the following headings, and in the order 
set forth below (37 C.F.R. §41. 37(c)): 

I. Real Party in Interest 

n. Related Appeals and Interferences 

III. Status of Claims 

IV. Status of Amendments 
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V. Summary of Claimed Subject Matter 

VI. Grounds of Rejection to be Reviewed on Appeal 

VII. Arguments 

□ Argument VIIA. Rejections Under 35 U.S.C. § 1 12, first 

paragraph 

□ Argument VIIB. Rejections Under 35 U.S.C. §112, second 

PARAGRAPH 

□ Argument VIIC. Rejections Under 35 U.S.C. § 102 
0 Argument VIID. Rejections Under 35 U.S.C. § 103 

□ Argument Vim. Rejection Other Than 35 U.S.C. §§102, 103 

AND 112 

VIII. Claims Appendix 

IX. Evidence Appendix 

X. Related Proceedings Appendix 
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I. Real Party in Interest 

The real party in interest in the appeal is: 

□ the party named in the caption of this brief. 

0 the following party: NEC Corporation of Tokyo, Japan 
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II. Related Appeals and Interferences 

With respect to other appeals, interferences or judicial proceedings that will 
directly affect, or be directly affected by, or have a bearing on the Board's decision in 
this appeal: 

0 there are no related appeals, interferences or judicial proceedings related to, 
which directly affect or may be directly affected by or have a bearing on the Board's 
decision in this pending Appeal. 

□ these are as follows: 
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III. Status of Claims 

The status of the claims in this application are: 

A. Total number of claims in Application 

Claims in the application are: Claims 1 to 12 

B. Status of all the claims: 

1 . Claims cancelled: None 

2. Claims withdrawn from consideration but not cancelled: None 

3. Claims pending: Claims 1 to 12 

4. Claims allowed: None 

5. Claims rejected: Claims 1 to 12 

C. Claims on Appeal. 

The claims on appeal are: Claims 1 to 12 
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IV. Status of Amendments 

The status of amendments filed subsequent to the rejection mailed July 14, 
2006, are as follows: 

There are no such amendments. 
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V. Summary of Claimed Subject Matter 

The invention is directed to real time recording/reproducing systems for 
recording and real time (instantaneously) reproducing image (or video) data. The 
invention solves a specific problem of such systems that make use of personal 
computer (PC) software processing. Performance insufficiency of the central 
processing unit (CPU) of the PC may arise when the system is operated in parallel 
with other applications or at the moment of starting the same. In consequence, it 
becomes impossible to obtain full frame real time capturing, real time compression 
and real time decompression. This leads to missing of frame data in the compression 
processing and delay in the reproducing processing and also to a further problem of 
deviation from synchronism of image and voice to each other. 

As shown in Figure 1, an analog-to-digital converter (ADC) 102 receives an 
analog image signal from an image input terminal 101. The output from the ADC 102 
is provided to a frame memory 103, followed by a compression processing module 
104, and dien a recorder 105. A digital signal read from the recorder 105 is input to a 
decompression processing module 106, followed by a frame memory 107 and a 
digital-to- analog converter (DAC) 108, so that a reproduced video signal is output at 
output terminal 109. A frame rate controller 1 10 executes a frame interpolation 
process such as to provide a constant frame rate of reading of data from the frame 
memory 103 to the compression processing module 104. The compression processing 
module 104 executes digital compression processing in a compressing system, which 
conforms to, for instance, MPEG (Motion Picture Experts Group) standards. The 
compression and decompression processing modules 104 and 106 constitute CPU 
software processing parts, the ADC 102 may be a video capture card, the frame 
memories 103 and 107 may be, for example, memories such as main memory and 
video memory, the DAC 108 may be a graphic accelerator card, and the recorder 105 
may be constituted by a hard disc drive or the like. 
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When the system becomes unable to execute full frame real time processing 
due to the CPU performance insufficiency, the frame rate controller 110 executes a 
control process of controlling the frame rate of data read out from the frame memory 
103 to the compression processing module 104 to be constant . When the compression 
processing module 104 becomes unable to execute full frame real time compression 
processing, it executes a thirming-out process, in which compression processing on 
some frames is omitted, thus generating digital compressed data at a substantially 
reduced frame rate while retaining a fixed standard frame rate . The decompression 
processing module 106 has a frame skipping function of skipping some frames in the 
decompression processing for reproduction synchronous to voice data with omitting a 
part of the frame decompression process. The frame rate controller 1 10 holds a 
constant intrinsic frame rate at all times by executing the frame interpolating 
processing in the event when the system becomes unable to execute full frame real 
time frame capturing. The manner in which this is done is described in detail at pages 
10-18 of the specification with reference to Figures 2-9. 

When the ADC 102 becomes unable to execute full frame real time capturing 
process due to the CPU performance insufficiency, the frame data stored in the frame 
memory 103 becomes discontinuous such that some frames are dropped out. Figure 2 
shows an example of such drop-out. In this example, frames (n + 1), (n + 4) and 
(n + 5) are dropped out. The frame rate controller 1 10 executes frame drop-out 
judgment by obtaining time data of each captured frame from the ADC 102. The 
controller 1 10 then controls the frame rate of the input to the compression processing 
module 104 to be constant by executing a frame interpolation process concerning the 
dropped-out frames. The controller 1 10 normally designates a pointer of a memory 
area with each frame data stored therein to the compression processing module 104. 
However, in the case of frame drop-out generation as shown in Figure 2, for the 
compression with respect to the frame (n + 1) the frame rate controller 1 10 designates 
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pointer pO of preceding compressed frame (n) once again. Likewise, for the 
compression with respect to the frames (n + 4) and (n + 5) the controller 110 
designates pointer p2 of the frame (n + 3) twice continuously. In this way, the 
controller 1 10 controls the frame rate in the compression processing module 104 to be 
constant. As an alternative to the above continuously pointer designating method, it is 
possible to adopt the following method. When designating pointer pi of the frame 
(n + 2) , the controller 110 separately informs that one frame has been dropped out. 
Likewise, when designating pointer p3 of the frame (n + 6), the controller 1 10 
separately informs that two frames have been dropped out. The actual frame 
interpolating process may be executed in the compression processing module 104. 

In the real time recording/reproducing system shown in Figure 1, the 
compression processing module 104 executes digital compression processing in a 
compressing system, which conforms to, for instance, MPEG standards. When the 
module 104 becomes unable to execute full frame real time compression processing, 
it executes frame thinning-out processing for reducing the actual frame rate by 
thinning out some frames in the compression processing while holding a fixed regular 
frame rate as prescribed in the MPEG standards. The CPU load in the compression 
processing is thus reduced so as to be able to continue the real time processing. In the 
MPEG standards, three different "frame types", i.e., types of frames subjected to the 
compression processing, are defined, that is, with omitting a part of the frame 
compression process "I frame" which does not require any reference frame and may 
serve as a reference frame for other frames, "P frame" which requires a reference 
frame and may also serve as a reference frame for other frames, and "B frame" which 
requires a reference frame and does not serve as any reference frame. The frame 
thinning-out processing is executed with respect to "B frames", which are always 
non-reference frames among the above three different frame types. 
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The procedure of thinning out input frames is illustrated in Figures 3(a) to 
3(c). Figure 3(a) shows an "input frame sequence". It is shown that I frame #1, B 
frames #2 and #3, P frame #4, B frames #5 and #6, and P frame #7 are compressed. 
Figure 3(b) shows a "frame compression order". The I frame #1 and P frame 10 #7 
which have been compressed earlier, are used as reference frames for the B frames #2 
and #3. Likewise, the P frames #4 and #7 having been compressed earlier are used as 
reference frames for the B frames #5 and #6. Figure 3(c) shows a "compression bit 
stream" when thinning out the frames #2 and #6. For the frames #1, #4, #3, #7 and #5, 
a picture header 81 representing the start of a frame compression code is added to the 
bit stream. The data subsequent to the picture header is then coded, and the coded 
data is added to the bit stream. The frames #2 and #6 are processed likewise until the 
addition of the picture header 81 representing the start of frame compression code to 
the bit stream. In these frames, however, the data subsequent to the picture header 81 
are not coded. Instead, code 83 representing the same frame as the reference frame is 
added to the bit stream. The same frame as the reference frame is constituted by an 
adjacent reference frame in the input frame sequence. That is, in frame #2 the added 
code 83 represents that this frame is the same as the immediately preceding frame #1, 
and in the frame #6 it represents that this frame is the same as the immediately 
succeeding frame #7. These rules are absolute rules. This frame is the same as the 
immediately preceding frame, and as the immediately succeeding frame. Thus, the 
codes may be stored in a main memory or the like, so that only codes read out from 
the memory may be added without any coding processing. Figure 3(d) shows a "frame 
decompression order", in which the compression bit stream is decompressed. It is 
shown that the actual frame rate is reduced while holding the total input frame rate, 
i.e., the MPEG standard frame rate, to be constant. 

The case of execution of the frame interpolating processing in the frame rate 
controller 110 and the frame thinning-out processing in the compression processing 
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module 104 independently of each other is illustrated in Figures 4(a) to 4(c). Figure 
4(a) shows an input frame sequence in the case when frames #2 and #5 are thinned 
out by the frame rate controller 1 10. Figure 4(b) shows a compression frame order, 
which is obtained with the input of frames #3 and #6 as interpolated frames, obtained 
as a result of interpolation with frames #1 and #4 (shown as frames #1' and #4'), 
respectively, to the compression processing module 104, are thiimed out in the frame 
thirming-out processing therein. Figure 4(c) shows a decompression frame sequence 
obtained by subsequent compression bit stream decompression, hi this case, only 
three frames, i.e., frames #1, #4 and #7, involve motion. Figure 5(a) shows an input 
frame sequence in the case of frame thinning-out in the compression processing 
module 104. The frame rate controller 1 10 checks whether frame interpolating 
processing has been performed. Figure 5(b) shows a compression frame order 
obtained as a result of frame thinning-out preferentially from frames obtained by 
interpolation in the frame rate controller 1 10. Figure 5(c) shows a decompression 
frame sequence obtained as a result of compression bit stream decompression when 
the same two frames are interpolated. In this case, the frames involving motion are 
increased to five frames, i.e., frames #1, #3, #4, #6 and #7. When the frame rate 
controller 1 10 decides that a compression frame obtained by frame interpolation is a 
reference frame, the compression processing module 104 becomes unable to execute 
any frame thinning-out processing, hi such a case, the frame thinning-out is 
performed by substituting the immediately preceding B frame in the input frame 
sequence. Figures 6(a) to 6(c) show the input frame sequence, the compression frame 
order and the decompression frame sequence, respectively, in this case. As shown, the 
subject of the frame interpolation does not concern the frame #3' obtained by the 
frame interpolation but the immediately preceding B frame #3. 

Referring to Figure 1, the decompression processing module 106 executes 
digital decompression processing in a decompressing system, which conforms to the 
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MPEG standards, for instance. The module 106 reads out the compression bit stream 
data recorded in the recorder 105 for decompression processing in a given time. The 
decompressed frame data is accumulated in the frame memory 107. The frame data 
accumulated in the frame memory 107 is converted in the DAC 108 to analog form, 
and then outputted via the output terminal 109 to an external display terminal for 
display. When the decompression processing module 106 becomes unable to execute 
full frame real time decompression processing, a check is performed as to the delay 
with respect to voice. When a status involving a delay takes place, frame skipping 
control of omitting the compression bit stream decompressing processing is 
performed. The control is performed such as to continue reproduction synchronous to 
voice. The case of execution of the frame thinning-out processing in the compression 
processing module 104 and the frame skipping processing in the decompression 
processing module 106 independently of each other is illustrated in Figures 7 to 9. 
Figure 7(a) shows a compression frame order, which is adopted in the thinning-out of 
frames #3 and #6 in the compression processing module 104. Figure 7(b) shows a 
decompression frame sequence obtained as a result of the skipping of two frames, i.e., 
frames #5 and #8. hi this case, only six frames, i.e., frames #1, #2, 5 #4, ##7, #9 and 
#10, involve motion. Accordingly, the compression processing module 104 adds bit 
stream data as shown in Figure 8. Specifically, when executing the frame 
thinning-out, the module 104 inserts user data 82, clearly representing that the 
pertinent frame has been thinned out, between the picture header 8 1 and the code 83 
representing that the frame is the same frame as the reference frame. Thus, when 
reproducing the bit stream data recorded in the recorder 105, the decompression 
processing module 106 can clearly determine that the pertinent frame has been 
thinned out. When the decompression processing module 106 decompresses 
compression bit stream data read out from the recorder 105, it checks whether the user 
data 8 1 representing that the pertinent frame has been thinned out is present right after 
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the picture header 82 representing the start of frame compression code. Then the 
module 106 finds that the pertinent frame has been thirmed out, aijd frame skipping 
control is performed with preferential omitting of the decompression processing. 
Figure 9(a) shows a compression frame order in such a case. Here, frames #4' and #7' 
have been thinned out in the decompression processing module 104. For these frames, 
the user data 82 representing that the pertinent frame has been thinned out, is added 
subsequent to the picture data 81. Figure 9(b) shows a decompression frame 
sequence. The module 106 checks whether the user data 82 is present right after the 
picture data 81, and executes frame skipping processing preferentially from the 
frames, which the user data 82 is added for. Thus, when the same two frames are 
skipped, the frames involving motion are increased to eight frames, i.e., frames#l, #2, 
#4, #5 and #7 to #10. 

The system described is recited in independent claims 1, 2 and 3, and the 
method performed by that system is recited in claim 6. More particularly, with 
reference to Figure 1, claim 1 recites a "real time recording/reproducing system for 
converting an analog image [101] in an analog-to-digital converter (ADC) [102] to 
digital data in a recorder [105], reading out the digital data recorded in the recorder 
and converting the read-out digital data in a digital-to-analog converter (DAC) [108] 
to analog data [109] to be outputted". The claimed real time recording/reproducing 
system includes "a first frame memory [103] for storing the output of the ADC; a 
compressing processing module [104] for compressing the output of the first frame 
memory; a decompression processing module [106] for decompressing the digital 
data read out from the recorder; a second frame memory [107] for storing the output 
of the decompression processing module and outputting the stored data to the DAC; 
and a frame rate controller [1 10] /or controlling the compression processing module 
to hold a constant intrinsic frame rate by executing frame interpolating processing" 
(emphasis added). Claim 2 recites that the frame rate controller 1 10 controls "the 
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frame rate of the compression processing module to be constant by executing a frame 
interpolating processing, and wherein the compression processing module has a 
frame thinning-out function of reducing an actual frame rate while holding a constant 
intrinsic frame rate" (emphasis added). Claim 3 recites that the frame rate controller 
1 10 controls "the compression processing module to be constant by executing frame 
interpolating processing, wherein the decompression processing module has a 
function of thinning out frames for continuing reproduction synchronous to voice" 
(emphasis added). 

Claims 4, 5 and 9 to 12 are either directly or indirectly dependent on claim 1. 
Claim 4 adds that "frame thinning-out in the decompression processing module and 
the frame skipping in the decompression processing module are performed 
preferentially from frame-interpolation frames to generate digital compressed data 
involving such motion'' (emphasis added). Claim 5 adds that "the compression 
processing module adds data bit stream data including a picture header representing 
a start of a frame compression code, a user data representing a thinned-out frame 
and a reference frame code representing the same frame as a reference frame" 
(emphasis added). 

As mentioned, the preferred embodiment of the invention involves the use of a 
personal computer (PC). Claim 9 specifically recites the "the compression processing 
module and the decompression processing modules are constituted by central 
processing unit (CPU) software processing units'" (emphasis added). Claim 10 builds 
on this by reciting that ''the ADC is a video capture card and the DAC is a graphic 
accelerator card' (emphasis added). Claim 1 1 follows this with the addition that ''the 
first and second frame memories are main memory and video memory, respectively 
(emphasis added). And claim 12 completes this by reciting that "the recorder is 
constituted by a hard disc drive" (emphasis added). 

As mentioned, claim 6 recites the method performed by the system and 
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includes the steps of "controlling the frame rate of the compressed output of the first 
frame memory to be constant by executing frame interpolating processing; and 
executing frame skipping processing when full frame real time decompression 
processing cannot be executed" (emphasis added). Claims 7 and 8 are dependent on 
claim 6. Claim 7 adds that ''the frame thinning-out and the frame skipping operations 
are performed preferentially from frame-interpolation frames to generate digital 
compressed data involving much motion" (emphasis added). Claim 8 adds that "in the 
compression processing operation data bit stream data including a picture header 
representing the start of a frame compression code, a user data representing thinned- 
out frame and a reference frame code representing the same frame are added as a 
reference frame" (emphasis added). 

As has been described in the foregoing, the real time recording/reproducing 
system according to the present invention has the following practically pronounced 
effects. Firstly, the system can continually execute real time image recording 
processing even when it becomes unable to execute full frame real time recording 
processing due to CPU performance insufficiency, which may arise when the system 
is operated in parallel with other applications or at the moment of starting the same. 
This is so because the system comprises the frame rate controller, which can hold a 
constant intrinsic frame rate at all times by executing the frame interpolating 
processing in the event when the system becomes unable to execute full frame real 
time frame capturing, and also the compression processing module, which has a frame 
thinning-out function of reducing the actual frame rate while holding the constant 
intrinsic frame rate. 

Secondly, even when the system becomes unable to execute full frame real 
time reproducing processing due to CPU performance insufficiency, it can perform 
time shift reproduction in a given time while holding synchronism with voice. This is 
so because the system comprises the decompression processing module, which has 
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the function of thinning out frames for continuing reproduction synchronous to voice. 

Thirdly, the system can record and reproduce compressed data involving much 
motion even when it becomes unable to execute full frame real time reproducing 
processing due to CPU performance insufficiency. This is so because of the fact that 
the compression processing module has the function of executing frame thinning-out 
processing preferentially from frames obtained by frame interpolation and also that 
the decompression processing module has the function of 
15 executing frame skipping processing preferentially from thinned-out frames. 
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VI. Grounds of Rejection to be Reviewed on Appeal 

Claims 1 to 12 stand rejected under 35 U.S.C. § 103(a) as follows: 

Claims 1 and 9 to 12 are rejected as being unpatentable over U.S. Patent No. 

5,966,495 to Takahashi et al. in view of U.S. Patent No. 6,163,646 to Tanaka et al. in 

further view of U.S. Patent No. 6,608,938 to Honda et al. in further view of U.S. 

Patent No. 6,404,928 to Shaw et al. 

Claims 2, 3, 4, 6, and 7 are rejected as being unpatentable over U.S. Patent 

No. 5,966,495 to Takahashi et al. in view of U.S. Patent No. 6,163,646 to Tanaka et 

al. in further view of U.S. Patent No. 6,608,938 to Honda et al. in further view of U.S. 

Patent No. 6,404,928 to Shaw et al. in further view of U.S. Patent Application 

Publication US 2004/0240744) to Honda et al. 

Claims 5 and 8 are rejected as being unpatentable over U.S. Patent No. 

5,966,495 to Takahashi et al. in view of U.S. Patent No. 6,163,646 to Tanaka et al. in 

further view of U.S. Patent No. 6,608,938 to Honda et al. in further view of U.S. 

Patent No. 6,404,928 to Shaw et al. in further view of U.S. Patent Application 

Publication US 2004/0240744) to Honda et al. in further view of U.S. Patent No. 

6,697,566 to Fujinami et al. 

The foregoing rejections are the same rejections made in the final rejection of 

the Office Action mailed January 11, 2006, but with the addition of the patent to 

Shaw et al. in each case. 
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Argument VIIA. Rejections Under 35 U.S.C. §112, first paragraph 
There are no rejections under 35 U.S.C. §1 12, first paragraph. 
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Argument VIIB. Rejections Under 35 U.S.C. § 1 12, second paragraph 
There are no rejections under 35 U.S.C. §112, second paragraph. 



Docket 2000P1 20495 (02530022AA) 
S.N. 09/837,190 



Argument VIIC. Rejections Under 35 U.S.C. §102 
There are no rejections under 35 U.S.C. §102. 
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Argument VIE). Rejections Under 35 U.S.C. §103 

The Examiner has failed to make out a prima facie case of obviousness under 
35 U.S.C. §103 but has, instead, relied on hindsight in an attempt to piece together 
several disparate references to arrive at his conclusion of obviousness. Specifically, 
the Examiner rejects claims 1 and 9 tol2 as being unpatentable over U.S. Patent No. 
5,966,495 to Takahashi et al. in view of U.S. Patent No. 6,163,646 to Tanaka et al. in 
further view of U.S. Patent No. 6,608,938 to Honda et al. in further view of U.S. 
Patent No. 6,404,926 to Shaw et al. The Examiner rejects claims 2, 3, 4, 6, and 7 as 
unpatentable over the patents to Takahashi et al. ('495) in view of Tanaka et al. 
('646), Honda et al. ('938), Shaw et al. ('926) in further view of U.S. Patent 
Application Publication US 2004/0240744 to Honda et al. The Examiner rejects 
claims 5 and 8 as unpatentable over Takahashi et al. ('495) in view of Tanaka et al. 
('646), Honda et al. ('938), Shaw et al. ('926), Honda et al. ('744), further in view of 
U.S. Patent No. 6,697,566 to Fujinami et al. 

The patents to Takahashi et al. ('495), Tanaka et al. ('646), Honda et al. 
('938), and Shaw et al. ('926) are common to each of the three rejections and are 
discussed first below. 

Takahashi et al. disclose a recording and reproducing apparatus which 
includes a recording circuit for forming predetermined memory units by correlating 
video information and audio information with retrieval information for retrieval of the 
video information and the audio information, and sequentially recording the 
predetermined memory units in a solid-state memory device at intervals of 
predetermined time, a file generating circuit for generating a retrieval information file 
for retrieving the video information and the audio information on the basis of the 
retrieval information, and a reproducing circuit for reading out the video information 
and the audio information from the solid-state memory device on the basis of 
information contained in the retrieval information file and reproducing a video signal 
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and an audio signal from the solid-state memory device, during retrieval of 
information recorded in the solid-state memory device. (See Abstract.) 

Takahashi et al. address a specific problem found in video tape recorders 
(VTRs). In such devices, a rotary head is provided on a rotary drum to record and 
reproduce video information for one field per track on and from magnetic tape 
wrapped aroimd the rotary drum. Because the video information is recorded on tape 
recording medium, time is needed to transport tape to retrieve desired video 
information. Also, because the rotary head makes contact with the magnetic tape on 
which information is recorded, the magnetic tape may be abraded or scratched, 
affecting the reliability of the recorded information. ]n addition, since the recorded 
information is based on an analog signal, if the information is copied to another 
medium, there is a degradation of image quality. 

To solve these problems, Takahashi et al. provide a recording and reproducing 
apparatus which comprises recording means for forming predetermined memory units 
by correlating video information and audio information with search information for 
search of the video information and/or the audio information, and sequentially 
recording the predetermined memory units in a memory device at intervals of 
predetermined time, file generating means for generating a search information file for 
searching the video information and/or the audio information on the basis of the 
search information and generating a search data file for searching the video 
information and/or the audio information in accordance with a variation in 
information indicative of an operational state of the recording and reproducing 
apparatus, and reproducing means for reading out the video information and/or the 
audio information from the memory device on the basis of information contained in 
the search information file and the search data file and reproducing a video signal 
and/or an audio signal from the memory device, during search of information 
recorded in the memory device. 
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Takahashi et al. show their recording and reproducing apparatus in Figure 1. 
This apparatus includes input terminals 1 and 2 through which to input a left (L) 
channel audio signal and a right (R) channel audio signal, respectively, an input 
terminal 3 through which to input a video signal, A/D (analog-to-digital) converters 4 
and 5 for converting the left channel and right channel signals into digital signals, 
respectively, and an A/D converter 6 for converting the video signal into a digital 
signal. The apparatus also includes audio signal processing circuits 7 and 8 for 
applying predetermined processing to the left channel and right channel digital audio 
signals, respectively, a video signal processing circuit 9 for applying predetermined 
processing to the digital video signal, an audio data compressing circuit 10 for 
performing compression processing of the left channel and right channel audio data 
processed by the respective audio signal processing circuits 7 and 8, and a video data 
compressing circuit 1 1 for performing compression processing of the video data 
processed by the video signal processing circuit 9. The apparatus also includes an ED 
generating circuit 12 for generating retrieval information (ID data) for data retrieval 
on the basis of a time code, memory information and the like as will be described 
later, a data combining circuit 13 for generating a data block for each ID data from the 
compressed audio data, the compressed video data and the ID data, and a main 
memory 18 which serves as a recording medium in which to record the data block and 
which is made up of solid-state devices. The apparatus also includes a memory 
controller 17 for controlling reading from and writing to the main memory 18, a 
system controller 16 for controlling the memory controller 17 and the entire 
apparatus, a calendar clock generating circuit 14 for generating a time code such as 
year, month, day, hour, minute, second and the like, a memory information generating 
circuit 15 for generating memory information indicative of the amount of data 
contained in information stored in the main memory 18, operating keys 19 to be 
operated by a user, and a display device 20 for displaying the required information 
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such as an operating mode which is selected in the apparatus. The apparatus also 
includes a data distributing circuit 21 for distributing the data block read from the 
main memory 18 into ID data, video data and audio data, a retrieval information 
reproducing circuit 22 for reproducing the ID data from the distributed data, a display 
information generating circuit 23 for reproducing display information from the ID 
data, a video data expanding circuit 24 for performing expansion processing of the 
distributed video data, an audio data expanding circuit 25 for performing expansion 
processing of the distributed audio data, and a data selecting circuit 26 for 
synchronously selecting the expanded video data and audio data. The apparatus also 
includes an adder 27 for adding together the display information generated by the 
display information generating circuit 23 and the video data selected by the data 
selecting circuit 26, a D/A converter 28 for converting the addition output of the adder 
27 into an analog signal, a video monitor 30 which is used for monitoring a video 
signal, D/A (digital-to-analog) converters 29-L and 29-R for converting the left and 
right channel audio data selected by the data selecting circuit 26 into analog signals, 
audio monitors 31-L and 31-R which are used for monitoring left and right channel 
audio signals, an output terminal 32 for the video data, and an output terminal 33 for 
the audio data. 

The operation of the recording and reproducing apparatus is as follows. A left 
(L) channel audio signal and a right (R) channel audio signal which form a 
stereophonic audio signal are respectively inputted through the audio signal input 
terminals 1 and 2, and are converted into digital audio signals by the A/D converters 4 
and 5, respectively. These digital audio signals are respectively subjected to 
predetermined processing, such as various kinds of noise reduction and restriction of 
their dynamic ranges, in the audio signal processing circuits 7 and 8. The 
thus-processed digital audio signals are subjected to data compression processing in 
the audio data compressing circuit 10. As the data compression processing, it is 
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possible to employ, for example, a coding method proposed by MPEG (Moving 
Picture Expert Group), such as adaptive conversion coding (AT AC, ASPEC) or band 
division coding (MUSICAM, SB/ADPCM), or vector coding of two right and left 
channels which are mixed by using a L/R correlation. In the meantime, a video signal 
which is inputted through the video signal input terminal 3 is converted into a digital 
signal by the A/D converter 6 capable of performing higher-speed processing than the 
A/D converters 4 and 5. The digital video signal is, as required, subjected to signal 
processing for conversion into signal components which will be described later in the 
video signal processing circuit 9. Then, the amount of data of the digital video signal 
is compressed to approximately one several-tenth (one over several tens) to 
approximately one several-hundredth (one over several hundreds) in the video data 
compressing circuit. This data compression processing can be realized by 
appropriately combining various kinds of processing, such as interframe or interfield 
correlation processing which employs an interframe or interfield image correlation, 
and motion-vector compensation for reducing image degradation involved in the 
interframe or interfield correlation processing, and both-side predictive interframe 
compression executed on the basis of previous and subsequent (past and future) 
frames taken along a time axis. As one example, according to what is commonly 
called MPEG-2 which is an MPEG-proposed algorithm, it is possible to ensure a 
standard image quality approximately equivalent to the image quality of a so-called V2 
VTR. When the data compression processing is completed, the ID data outputted 
from the ID generating circuit 12 which will be described later and the aforesaid 
compressed audio data and video data are combined into a data block which 
constitutes a predetermined memory unit, in the data combining circuit 13. The data 
block is sequentially stored m the main memory 18. The main memory 18 is a 
large-capacity solid-state circuit made up of a flash memory, a DRAM, an SRAM or 
the like, and the memory controller 17 performs control of the memory addresses of 



Docket 2000P1 20495 (02530022 AA) 
S.N. 09/837,190 

26 



the main memory 18, writing to and reading from the main memory 18 and the like. 
The memory controller 17 is controlled by the system controller 16. The system 
controller 16 performs not only control of the aforesaid memory controller 17 but also 
control of the entire apparatus, such as control for selecting an operation from among 
recording, reproduction and retrieval. Specifically, when the system controller 16 
receives an instruction, such as a recording instruction, a reproduction instruction or a 
retrieval instruction, inputted by an operation of the operating keys 19, the system 
controller 16 performs control of the memory controller 17 and simultaneously causes 
the display device 20 to display the operating status of the apparatus and information, 
such as a time code indicative of a recording/reproduction time. The system controller 
16 also transmits the information, such as the time code, to the memory information 
generating circuit 15. The aforesaid time codes are mainly divided into two kinds. The 
first kind of time code represents information indicative of an elapsed time which 
passes after the start of the main memory 18 or a video program and the cumulative 
time of camera photography, and the second kind of time code represents information 
indicative of the year/month/day and the instant of time of recording or camera 
photography. To generate the latter kind of time code, the calendar clock generating 
circuit 14 is provided. The memory information generating circuit 15 receives data, 
such as data indicative of the status of storage of information in the main memory 18, 
from the memory controller 17, and supplies the received data to the ID generating 
circuit 12 as memory information indicative of the amount of data contained in the 
information recorded in the main memory 18. The contents of the memory 
information include, for example, data indicative of the time codes and selected 
modes for image quality and sound quality, which are based on the information 
supplied from the system controller 16, the amount of video data or audio data which 
is based on the information supplied from the memory controller 17 (a data length, in 
the case of variable length coding), and the value of a starting address of a data 
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storage location in the main memory 18. The ID generating circuit 12 generates ID 
data from the memory information and the information supplied from the system 
controller 16, and supplies the ID data to the data combining circuit 13. The data 
combining circuit 13 combines the ID data and the compressed audio data and video 
data into a data block which constitutes a predetermined memory unit. The data block 
is sequentially stored in the main memory 18 for each ID data. The starting address of 
each data block stored in the main memory 18 is sequentially written into an ID file 
set in the main memory 18. 

Takahashi et al. neither recognize nor solve the problem solved by the claimed 
invention, to wit, performance insufficiency of the CPU when the system is operated 
in parallel with other applications or at the moment of starting the same and, in 
consequence, it becomes impossible to obtain full frame real time capturing, real time 
compression and real time decompression, leading to missing of frame data in the 
compression processing and delay in the reproducing processing and also to a further 
problem of deviation from synchronism of image and voice to each other. 

Tanaka et al. disclose an apparatus for a synchronized playback of compressed 
digital data of audio-video signals with a simple configuration to be applied to a 
system having comparatively low data processing ability, independent of resolution of 
a software timer of the system, the apparatus of the invention obtains a reference time 
indicating reproduction time passage of the audio signal component from reproduced 
data amount of the audio signal component, and detects progress of the video signal 
component in reference to the reference time. 

The problem addressed by Tanaka et al. relates to the synchronized playback 
of audio- video (AV) signals. When compressed digital data of AV signals are read 
out from recording media such as a CD-ROM (Compact Disk Read Only Memory) or 
a hard disk device are decompressed before being represented through output devices 
such as a CRT monitor and a speaker, if the data processing ability of the apparatus is 
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not sufficiently high compared to the video signal size of the compressed digital data, 
the apparatus can not reproduce both the audio signal component and the video signal 
component in real time with which they are recorded originally, taking too much time 
for the data decompression processes. In consequence, the video signal component 
may be represented as a slow-motion film and the audio signal component is 
reproduced intermittently. 

Tanaka et al. show in Figure 1 a block diagram illustrating their apparatus for 
a synchronized playback of AV signals. This apparatus comprises a data recording 
device 1 for storing compressed digital data of AV signals, an AV separation unit 2 
for extracting an audio signal component and a video signal component multiplexed 
in the compressed digital data read out of the data recording device 1, a video signal 
processor 3 for performing decompression of the video signal component extracted by 
the AV separation unit 2, an audio signal processor 4 for performing decompression 
of the audio signal component extracted by the AV separation unit 2, an AV 
synchronization controller 5 for performing synchronization control between the 
audio signal component and the video signal component, a CRT monitor 6 for 
displaying the video signal component, and a speaker 7 for reproducing the audio 
signal component. The video signal processor 3 includes a compressed video data 
buffer 1 1 for storing compressed data of the video signal component extracted by the 
AV separation unit 2, a video decoder 12 for decompressing the compressed data of 
the video signal component buffered in the compressed video data buffer 11 by 
analyzing attribute information (video header information) thereof, a frame buffer 13 
for buffering frame data of the video signal component decompressed by the video 
decoder 12, and a video display controller 14 for generating display signals to be 
supplied to the CRT monitor 6 by reading out the frame data prepared in the frame 
buffer 13 in synchronization with synchronous signals for the CRT monitor 6. The 
audio signal processor 4 includes a compressed audio data buffer 21 for storing 
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compressed data of the audio signal component extracted by the AV separation unit 2, 
an audio decoder 22 for decompressing the compressed data of the audio signal 
component buffered in the compressed audio data buffer 21 by analyzing attribute 
information (audio header information) thereof, a PCM buffer 23 for storing data of 
the audio signal component decompressed by the audio decoder 22, and an audio 
reproduction controller 24 for generating sound signals to be supplied to the speaker 7 
according to the data of the audio signal component supplied from the PCM buffer 
23. The AV synchronization controller 5 includes a video frame counter 31 for 
counting and maintaining a real frame number of the video signal component which 
indicates a sum of a frame number actually decompressed and a frame number 
omitted to be decompressed by the video decoder 12, from the beginning of 
reproduction, an audio data counter 32 for counting and maintaining data amount of 
the audio signal component actually decompressed and reproduced by the audio 
reproduction controller 24 from the beginning of reproduction according to 
reproduction information supplied from the audio reproduction controller 24, a clock 
generator 33 for calculating a reference time indicating reproduction time passage of 
the audio signal component from the data amount counted by the audio data counter 
32 referring to the audio header information analyzed by the audio decoder 22, a delay 
detector 34 for calculating an ideal frame number of the video signal component to be 
displayed originally from the reference time calculated by the clock generator 33 
referring to the video header information analyzed by the video decoder 12, and 
detecting lead or lag of the video signal component to the audio signal component by 
comparing the ideal frame number and the real frame number counted by the video 
frame counter 31, and a frame omission controller 35 for performing the frame 
omission when the video signal component is delayed from the audio signal 
component according to the lead or lag detected by the delay detector 34 referring to 
priority of frames to be decompressed, hi other words, the video frame counter 3 1 of 
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the AV synchronization controller 5 operates as means for counting the real frame 
number of the video signal component actually processed (decompressed or omitted), 
and the audio data counter 32 and the clock generator 33 operate as means for 
obtaining the reference time indicating reproduction time passage of the audio signal 
component, as will be described afterwards, independent of a software timer of the 
system which may not available for correct synchronization because of its insufficient 
resolution. Further, the delay detector 34 operates as means for calculating the ideal 
frame number of the video signal component to be decompressed and displayed 
originally in an interval indicated by the reference time, as well as operating as means 
for detecting progress of the video signal component in reference to the reference 
time, that is, detecting the lead or lag of the video signal component to the audio 
signal component by comparing the ideal frame number and the real frame number. 
And die frame omission controller 35 operates as means for restoring the lead or lag 
of display timing of the video signal component to reproduction timing of the audio 
signal component by indicating the frame omission or decompression waiting of the 
video signal component to the video decoder 12, according to the progress of the 
video signal component detected by the delay detector 34. Here, the frame omission is 
performed according to the order of priority of frames as will be described in the 
following paragraphs, and the decompression waiting is performed when a lead of the 
video signal component is detected by the delay detector 34 until the audio signal 
component overtakes the video signal component. 

Figure 2 of Tanaka et al. is a schematic diagram illustrating an example of 
data stream of the video signal component subjected to the frame omission performed 
by the frame omission controller 35. In the example, frame data of the video signal 
component are ranged according to an inter-frame prediction coding system such as 
the MPEG standard, for example. The data of the video signal component has a frame 
composition comprising a plurality of GOP (Group Of Pictures) data, each including 
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data of I (Intra), P (Predictive) and B (Bidirectionaily predictive) pictures. The 
I-picture, coded from intra-frame data, can be decompressed independently from itself 
without any key frame, and is used as a key frame to be referred to when following 
frames are decompressed. The P-picture, coded according to forward prediction, is 
decompressed referring to a preceding key frame. The P-picture is also used as a key 
frame to be referred to when following and preceding frames are decompressed. The 
B -picture, coded according to bi-directional prediction, is decompressed referring to a 
preceding key frame and a following key frame, being referred to by no other frame. 
The key frame is a picture to be referred to when other pictures are decompressed, and 
so, when a key frame is omitted to be decompressed, following P and B-pictures until 
a next I-picture appears should be also omitted, unable to be decompressed. 
Therefore, the frame omission priority is attributed to the B-picture, the P-picture and 
the I-picture, in that order. The PCM buffer 23 for storing the data of the audio signal 
component to be reproduced is described referring to a schematic diagram of Figure 3. 
The PCM buffer 23 operates as a ring buffer comprising a plurality of memory blocks 
(1) to (n), wherein the data of the audio signal component decompressed by the audio 
decoder 22 are stored to be read out and supplied to the audio reproduction controller 
24 in turns cyclically to be reproduced. When reproduction by the audio reproduction 
controller 24 of data read out and transferred from a memory block of the PCM buffer 
23 is accomplished, an interruption is occurred and data in next memory block is 
transferred to the audio reproduction controller 24 to be reproduced. The audio 
decoder 22 decompresses the audio signal component to be stored in the next memory 
block which is read out, interrupting processing of the video signal component even 
when the video signal component is under processing. For example, when 
reproduction of data of the memory block (1) of Figure 3 is accomplished, data of the 
memory block (2) is read out, transferred to the audio reproduction controller 24 and 
begins to be reproduced. In the memory block (1), data of the audio signal component 
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newly decompressed by the audio decoder 22 is stored to be read out after 
reproduction of data of the memory block (n) is accomplished. Thus, the audio signal 
component is decompressed with the interruption processing prior to other procedure 
such as the decompression process of the video signal component, in the embodiment. 
Therefore, the audio signal component can be reproduced without interruption, and 
accordingly, a precise reproduction time can be obtained from reproduction progress 
of the audio signal component at the audio reproduction controller 24. 

Thus, Tanaka et al. disclose an apparatus for a synchronized playback of 
compressed digital data of audio-video signals with a simple configuration to be 
applied to a system having comparatively low data processing ability, independent of 
resolution of a software timer of the system , whereas the apparatus of the claimed 
invention obtains a reference time indicating reproduction time passage of the audio 
signal component from reproduced data amount of the audio signal component, and 
detects progress of the video signal component in reference to the reference time . 

Honda et al. ('938) disclose an image data compression or expansion method 
and apparatus for use in a remote monitoring system which employs an image change 
detection means for detecting a change in image based on input image data and means 
for controlling a frame rate of outputted compressed image data. In other words, 
Honda et al. provide a variable frame rate depending on the rate of motion of an input 
image. This is quite different from what the claimed invention accomplishes. 
Specifically, the claimed invention aims to control the frame rate of the compression 
processing module to be constant by executing a frame interpolating processing. 
Clearly, there is no possibility of modifying the combined teachings of Takahashi et 
al. and Tanaka et al. with that of Honda et al. and arriving at the claimed invention. 

Shaw et al. disclose a system for producing a quantized scalable signal during 
a particular period of time in which the signal is digitized and is represented by a 
polynomial expression. The number of terms in the expression relates to the accuracy 



Docket 2000P1 20495 (02530022AA) 
S.N. 09/837,190 

33 



of the digitized signal. The digitized signals are compressed and then are transmitted, 
stored, retrieved and reconstructed. Shaw et al. allow for the access of a plurality of 
computing, consumer, and communication equipment, e.g., a personal computer (PC), 
a workstation (WS), camera, television, video cassette recorder (VCR), telephone, 
etc., and allow for conveying multiple types of media information, e.g., sound, image, 
animated graphics, and live video. The Shaw et al. system is dedicated to the specific 
application of teleconferencing. 

As shown in Figure 2 of Shaw et al., telephone 102, television 104, and 
personal computer 106 are physically interconnected by the box labeled "Our 
Invention", the purpose of which is to allow people to see each other face to face 
through television 104 or computer screen 105 when they are making voice phone 
calls. The box 112 also allows people to retrieve and review documents in real time 
from computer storage 101, send over the phone line 103 and display at the other end. 
The box 112 further allows TV studios to broadcast as many as 200,000 channels 
programs instead of 200 channels today. 

Shaw et al. provide scalable frame memory architecture reconfigurable 
techniques (SMART) described in Figure 15. The basic principle of SMART allows 
the host processor 314 shown in Figure 16 to identify types of input video image 
articles during the media import stage. The host processor instructs the 
reconfiguration circuit 1064 and the scaler circuit 1066 to provide the required down 
sampling ratio. The media article can then conform (reduce) to the internal file format 
during the importing stage. As appropriate, it also readjusts (enlarge) to another 
adequate external format during the exporting stage. Because of the smaller file size 
of the internal format, the real time performance requirement for the system hardware, 
i.e., pixel processor 306, graphics processor 1070, transform processor 308, motion 
processor 307, is much reduced. The size of the frame memory 312 is proportionally 
reduced. Since dedicated high speed hardware are no longer necessary, various coding 
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algorithms is internally microcoded at the pixel processor 306. A bandwidth 
controller 144 receives bandwidth requirement from the network communication 
processor 302. The bandwidth controller 144 then instructs the host processor 314 to 
develop the appropriate compression ratio in order to meet the real time performance 
requirement. Bandwidth controller 144 will also interface with the transmission 
processor 304 in order to import and export the media article at the appropriate 
bandwidth. As shown in Figure 8, the Shaw et al. system can program the network 
communication processor 302, transmission processor 304, and the display processor 
310 to provide the various types of communication interface. In Figure 10, there is 
shown the internal operation modes 315 for the host processor 314 to adapt different 
compression ratio in order to accommodate various network bandwidth requirement. 
As an example, there is listed in column 8 of the patent the following bandwidth 
requirements for some, of the popular network interface: 

a. Communicating over a analog phone line 532, whereby 9,600 bit per second 
bandwidth is required, a quarter common intermediate frame (QCIF) 151 
format is displayed at 7.5 frame per second; 

b. Communicating over a ISDN D channel 534 at 16 Kilo bits per second (Kps), 
The user has two options, either two quarter common intermediate frame 
(QCIF) 151 format is displayed at 7.5 frame per second (fps), or one QCIF 
frame 151 is displayed at 15 fps; 

c. Communicating over a analog phone line, whereby a 19,200 bit per second 
bandwidth is required. The user has two options, either two quarter common 
intermediate frame (QCIF) 151 format is displayed at 7.5 frame per second 
(fps), or one QCIF 151 frame is displayed at 15 fps; 

d. Communicating over switched 56 kilo bits per second (kps) digital network 
(PSDN) 537, QCIF 151 frames with 3 quality level options will be updated at 
15 fps 582; 



Docket 2000P1 20495 (02530022 AA) 
S.N. 09/837,190 

35 



e. Communicating over a single ISDN basic rate interface (BRI) B channels 538 
over a ISDN network, four QCIF 151 frames will be concurrently updated at 
15 fps 582; 

f. Communicating over a dual ISDN B channels 540 in a ISDN BRI network, 
QCIF 151 frames will be transmitted at 30 fps 200; 

g. Communicating over a 384 kps ISDN HI 542 network, CIF 149 frames will 
be transmitted at 15 fps 582; 

h. Communicating over a 1.544 kps Tl 544 network, CIF 149 frames will be 
transmitted at 30 fps 200. 

In other words, Shaw et al. are concerned with the management and control of 
variable communications bandwidth and address the flexible combinations of the 
digital coded multiple-media information fro a wide variety of application 
requirements, including teleconferencing, videophone and the like. 

The Examiner takes the position that "it would have been obvious to one of 
ordinary skill in the art at the time of the invention to modify the 
recording/reproducing apparatus, as disclosed by Takahashi et al, and further 
incorporate a system wherein a second frame memory is available for storing the 
decompressed data before outputting to the DAC, as disclosed by Tanaka et al., and 
further incorporate a frame rate controller for controlling compression, as disclosed by 
Honda et al, which holds a constant intrinsic frame rate by executing frame 
interpolating processing, as disclosed by Shaw et al." This position is taken without 
any indication as to why one of ordinary skill in the art would be disposed to 
undertake such a reconstruction, how one of ordinary skill in the art would actually 
make such a reconstruction, and what would result if such a reconstruction could 
actually be made. None of the references relied on by the Examiner address the 
problem solved by the claimed invention. Moreover, Honda ('938) provide a variable 
frame rate, and now the Examiner relies on Shaw et al. for some sort of teaching of 
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holding a "constant intrinsic frame rate". It would seem that the two are mutually 
exclusive. 

Claim 1 is directed to a "real time recording/reproducing system" which 
converts an analog image signal to in an analog-to-digital converter (ADC) to digital 
data, records the digital data in a recorder, reads out the digital data recorded in the 
recorder and converts the read-out digital data in a digital-to-analog converter (DAC) 
to analog data to be outputted. The foregoing is all recited in the preamble of the 
claim as the environment in which the claimed invention is implemented. The 
claimed invention, recited in the body of the claim, comprises "a first frame memory 
for storing the output of the ADC", corresponding to the frame memory 103; "a 
compression processing module for compressing the output of the first frame 
memory", corresponding to the compression processing module 104; "a 
decompression processing module for decompressing the digital data read out from 
the recorder", corresponding to the decompression processing module 106; "a second 
frame memory for storing the output of the decompression processing module and 
outputting the stored data to die DAC", corresponding to the frame memory 107; and 
"a frame rate controller for controlling the compression processing module to hold a 
constant intrinsic frame rate by executing frame interpolating processing " (emphasis 
added), corresponding to the frame rate controller 1 10. 

Claims 9 to 12 are either directly or indirectly dependent on claim 1 and, 
therefore, incorporate the same limitations. Claim 9 adds that "the compression 
processing module and the decompression processing module are constituted by 
central processing unit (CPU) software processing parts." Claim 10, dependent on 
claim 9, adds that the ADC is a video capture card and the DAC is a graphic 
accelerator card." As explained in the specification, these cards are cards typically 
found in a personal computer (PC). Claim 11, dependent on claim 10, adds that the 
first and second frame memories are main memory and video memory, respectively." 
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Again, these components are part of a PC. Finally, claim 12, dependent on claim 11, 
adds that "the recorder is constituted by a hard drive." And again, this is part of a PC. 

In rejecting claims 2, 3, 4, 6, and 7, the Examiner relies on Takahashi et al. 
('495), Tanaka et al. ('646), Honda et al. ('938), and Shaw et al. ('928) and adds U.S. 
Patent Application Publication US 2004/0240744 to Honda et al. Note that Honda et 
al. ('744) is a continuation of the application on which Honda et al. ('938) issued and 
therefore has the identical same disclosure. In making this rejection, the Examiner 
states that the combination of Takahashi et al. ('495), Tanaka et al. ('646), Honda et 
al. ('938), and Shaw et al. ('928) "fails to disclose the additional limitation of a frame 
rate controller for controlling the frame rate of the compression processing module to 
be constant by executing a frame interpolating process." The Examiner contends that 
Honda et al. ('744) "discloses an image data compression system wherein the frame 
rate of the compression module is controlled by the constant executing of frame rate 
interpolating process as seen in Figure 18 and further described in paragraphs 0025, 
0034, and 101-103." First, it is noted that paragraphs 0025 and 0034 are part of the 
"Summary of the Invention" and do not describe Figure 18. The Examiner's reference 
to "101-103" is not understood. In paragraph 0025, it is stated that "the input data is 
reproduced such that the frame rate is lower in a scene where the change between 
images is small than the frame rate in a scene where the change between images is 
large" (emphasis added). In other words, the frame rates are variable . In paragraph 
0034, it is stated that the method involves "decoding the compressed image data at the 
frame rate controlled to be lower than in the scene where the image change is small 
than in the scene where the image change is large . . ." (emphasis added). Again, the 
frame rates are variable . Second, Figure 1 8 is a block diagram showing the 
construction of the image expansion apparatus in relation to the image compression 
apparatus shown in Figure 17. The image compression apparatus of Figure 17 
includes a 
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". . . variable-length encoder 1705 variable-length encodes the image 
with the motion vector value. The synthesizing unit 1716 synthesizes 
the variable- length coded data with information indicating that the 
image has not been skipped, and if the previous image has been 
skipped, with information indicating that the previous image has been 
skipped, and outputs the synthesized data to the output terminal 1713. 
Further, information on time interval between the previously-sent 
image and the current image (e.g., information indicative of {fraction 
(1/30)} sec if the previous image has not been skipped, while 
indicative of {fraction (2/30)} sec if the image has been skipped) may 
be sent in place of the information indicating that the image has been 
skipped/has not been skipped. Note that in the local decoder 1707, 
similarly to the previous image, inverse quantization and inverse DCT 
transformation are performed on the difference image, and if an image 
added by the adder 1710 to the image motion-compensated by the 
motion compensation unit 1712 (i.e., the third image) is not skipped, 
the image is stored into the image memory 1711." (Paragraph 0122) 

The image expansion apparatus of Figure 18 includes a variable length decoder which 

". . . separates the image data into a motion vector and difference 
image data. The inverse quantizer 1803 and the inverse DCT unit 1804 
reproduce an original difference image from the difference image data. 
The motion compensation unit 1807 performs motion compensation on 
the first image in the image memory 1 806 in accordance with the 
motion vector from the variable-length decoder 1802. The adder 1805 
adds the motion-compensated image to the difference image, to 
reproduce the third image. The third image is stored into the image 
memory 1 806, and at the same time, is sent to the image interpolator 
1810. The image interpolator 1810 generates the second image by 
interpolation from the first and third images, if necessary, then outputs 
the second image to the output terminal 1808, and then outputs the 
third image to the output terminal 1808. The image interpolator 1810 
may generate one or more images by interpolation in accordance with 
information on skipped image." (Paragraph 0126) 

Claim 2 recites a real time recording/reproducing system which converts an 
analog image signal in an analog-to-digital converter (ADC) to digital data, records 
the digital data in a recorder, reads out the digital data recorded in the recorder and 
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converts the read-out digital data in a digital-to-analog converter (DAC) to analog 
data to be outputted. The foregoing is all recited in the preamble of the claim as the 
environment in which the claimed invention is implemented. The claimed invention, 
recited in the body of the claim, comprises "a first frame memory for storing the 
output of the ADC", corresponding to frame memory 103; "a compression processing 
module for compressing the output of the first frame memory", corresponding to 
compression processing module 104; "a decompression processing module for 
decompressing the digital data read out from the recorder", corresponding to 
decompression processing module 106; "a second frame memory for storing the 
output of the decompression processing module and outputting the stored data to the 
DAC", corresponding to frame memory 107; and "a frame rate controller for 
controlling the frame rate of the compression processing module to be constant by 
executing a frame interpolating processing, and wherein the compression processing 
module has a frame thinning-out function of reducing an actual frame rate while 
holding a constant intrinsic frame rate " (emphasis added), corresponding to frame rate 
controller 110. 

Claim 3 recites a real time recording/reproducing system which converts an 
analog image signal in an analog-to-digital converter (ADC) to digital data, records 
the digital data in a recorder, reads out the digital data recorded in the recorder and 
converts the read-out digital data in a digital-to-analog converter (DAC) to analog 
data to be outputted. The foregoing is all recited in the preamble of the claim as the 
environment in which the claimed invention is implemented. The claimed invention, 
recited in the body of the claim, comprises "a first frame memory for storing the 
output of the ADC", corresponding to frame memoryl03; "a compression processing 
module for compressing the output of the first frame memory", corresponding to 
compression processing module 104; "a decompression processing module for 
decompressing the digital data read out from the recorder and executing a frame 
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skipping processing when it becomes unable to execute full frame real time 
decompression processing " (emphasis added), corresponding to decompression 
processing module 106; "a second frame memory for storing the output of the 
decompression processing module and outputting the stored data to the DAC", 
corresponding to frame memory 107; and "a frame rate controller for controlling the 
compression processing module to be constant by executing frame interpolating 
processing, wherein the decompression processing module has a function of thinning 
out frames for continuing reproduction synchronous to voice " (emphasis added), 
corresponding to frame rate controller 110. 

Claim 4 is dependent on claim 1 and, therefore, incorporates all the limitations 
of claim 1, as discussed above. Claim 4 adds that "frame thinning-out in the 
decompression processing module and the frame skipping in the decompression 
processing module are performed preferentially from frame- interpolation frames to 
generate digital compressed data involving much motion." 

Claim 6 recites a real time recording/reproducing method which records a 
digital data in a recorder obtained by converting an analog image signal, and 
reproduces recorded digital data in an analog data format. The foregoing is all recited 
in the preamble of the claim as the environment in which the claimed invention is 
implemented. The steps of the claimed invention, recited in the body of the claim, 
comprise "storing the digital data in a first frame memory", wherein the first frame 
memory corresponds to frame memory 103; "compressing the output of the first 
frame memory", which step is performed by the compression processing module 106; 
"recording a compressed output of the first frame memory as digital data in the 
recorder", which step is performed by recorder 105; "decompressing the digital data 
read out from the recorder", which step is performed by decompression processing 
module 106; "storing the decompressed data in a second memory", wherein the 
second memory is frame memory 107; " controlling the frame rate of the compressed 



Docket 2000P1 20495 (02530022 AA) 
S.N. 09/837,190 

41 



output of the first frame memory to be constant by executing frame interpolating 
processing" (emphasis added), which step is performed by the frame rate controller 
1 10; and " executing a frame skipping processing when full frame real time 
decompression processing cannot be executed " (emphasis added), which step is also 
performed by the frame rate controller. 

Claim 7 is dependent on claim 6 and, therefore, incorporates all the limitations 
of that claim. Claim 7 adds that "the frame thinning-out and the frame skipping 
operations are performed preferentially from frame- interpolation frames to generate 
digital compressed data involving much motion." 

In rejecting claims 5 and 8, Examiner relies on Takahashi et al. ('495), Tanaka 
et al. ('646), Honda et al. ('938), Shaw et al. ('928), and Honda et al ('744) and adds 
U.S. Patent No. 6,697,566 to Fujinami et al. The Examiner seems to state that the 
combination of Takahashi et al. ('495), Tanaka et al. ('646), Honda et al. ('938), 
Shaw et al. ('928), and Honda et al ('744) does not disclose "a real time 
recording/reproducing system wherein the compression processing modules adds data 
bit stream data including a picture header representing the start of a frame 
compressing code, a user data representing a thinned-out frame and a reference frame 
code representing the same frame as a reference frame". The Examiner contends that 
Fujinami et al. ('566) "discloses a system wherein signals are encoded with the 
characteristic recording information and further more added to the data bit stream as 
disclosed in Column 26 Lines 50+ through Column 27 Lines 1-25." 

Fujinami et al. ('566) address the problem of recording pre-compressed 
MPEG files while maintaining management information for indicating sector 
positions, a process which normally required decoding and re-encoding followed by 
recording with a consequent loss of video quality. What Fujinami et al. do is to 
provide a characteristic pomt detection circuit which detects a characteristic point 
from a base band video signal and generate a characteristic point file. This 
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characteristic point file is multiplexed with the MPEG compressed video file which is 
recorded to optical disc and, during reproduction from the video disc, the 
characteristic point file is used to indicate sector positions for random access. 

Claim 5 is dependent on claim 1 and, therefore, incorporates the limitations of 
that claim. Claim 5 adds that "the compression processing module adds data bit 
stream data including a picture header representing a start of a frame compression 
code, a user data representing a thinned-out frame and a reference frame code 
representing the same frame as a reference frame." 

Claim 8 is dependent on claim 6 and, therefore, incorporates the limitations of 
that claim. Claim 8, similar to claim 5, adds that "in the compression processing 
operation data bit stream data including a picture header representing the start of a 
frame compression code, a user data representing a thinned-out frame and a reference 
frame code representing the same frame are added as a reference frame." 

Conclusion 

None of the references relied on by the Examiner show of suggest "a frame 
rate controller for controlling the compression processing module to hold a constant 
intrinsic frame rate by executing frame interpolating processing", as specifically 
recited in claim 1, for example. Therefore, no combination of the references can 
possibly suggest or teach this feature. The reason is simple - none of the references 
recognize, much less, solve the problem solved by the disclosed and claimed 
invention, which is real time recording and real time reproducing image or video data 
when the system becomes unable to execute full frame real time processing due to the 
CPU performance insufficiency. The Applicant solves this problem in a unique and 
elegant manner by providing a frame rate controller that executes a control process of 
controlling the frame rate of data read out from the frame memory to the compression 
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processing module to be constant , and when the compression processing module 
becomes unable to execute full frame real time compression processing, it executes a 
thinning-out process, in which compression processing on some frames is omitted, 
thus generating digital compressed data at a substantially reduced frame rate while 
retaining a fixed standard frame rate . Further, the decompression processing module 
has a frame skipping function of skipping some frames in the decompression 
processing for reproduction synchronous to voice data with omitting a part of the 
frame decompression process. The frame rate controller holds a constant intrinsic 
frame rate at all times by executing the frame interpolating processing in the event 
when the system becomes unable to execute full frame real time frame capturing. The 
Examiner's attempted hindsight reconstruction of multiple disparate references does 
not accomplish this result. 
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Argument Vim. Rejection Other Than 35 U.S.C. §§102, 103 and 1 12 

There are no rejections other than the rejections under 35 U.S.C. §103 
discussed above. 
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VIII. Claims Appendix 

The text of the claims involved in the appeal are: 



1 1. A real time recording/reproducing system for converting an analog image 

2 signal in an analog-to-digital converter (ADC) to digital data, recording the 

3 digital data in a recorder, reading out the digital data recorded in the recorder 

4 and converting the read-out digital data in a digital-to-analog converter (DAC) 

5 to analog data to be outputted, the real time recording/reproducing system 

6 comprising: 

7 a first frame memory for storing the output of the ADC; 

8 a compression processing module for compressing the output of the 

9 first frame memory; 

10 a decompression processing module for decompressing the digital data 

1 1 read out from the recorder; 

12 a second frame memory for storing the output of the decompression 

13 processing module and outputting the stored data to the DAC; and 

14 a frame rate controller for controlling the compression processing 

15 module to hold a constant intrinsic frame rate by executing frame interpolating 

16 processing. 

1 2. A real time recording/reproducing system for converting an analog image 

2 signal m an analog-to-digital converter (ADC) to digital data, recording the 

3 digital data in a recorder, reading out the digital data recorded in the recorder 

4 and convertuig the read-out digital data in a digital-to-analog converter (DAC) 

5 to analog data to be outputted, the real time recording/reproducing system 

6 comprising: 
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7 a first frame memory for storing the output of the ADC; 

8 a compression processing module for compressing the output of the 

9 first frame memory; 

10 a decompression processing module for decompressing the digital data 

1 1 read out from the recorder; 

12 a second frame memory for storing the output of the decompression 

13 processing module and outputting the stored data to the DAC; and 

14 a frame rate controller for controlling the frame rate of the 

15 compression processing module to be constant by executing a frame 

16 interpolating processing, and wherein the compression processing module has 

17 a frame thinning-out function of reducing an actual frame rate while holding a 

1 8 constant intrinsic frame rate. 

1 3. A real time recording/reproducing system for converting an analog image 

2 signal in an analog-to-digital converter (ADC) to digital data, recording the 

3 digital data in a recorder, reading out the digital data recorded in the recorder 

4 and converting the read-out digital data in a digital-to-analog converter (DAC) 

5 to analog data to be outputted, the real time recording/reproducing system 

6 comprising: 

7 a first frame memory for storing the output of the ADC; 

8 a compression processing module for compressing the output of the 

9 first frame memory; 

10 a decompression processing module for decompressing the digital data 

1 1 read out from the recorder and executing a frame skipping processing when it 

12 becomes unable to execute full frame real time decompression processing; 

13 a second frame memory for storing the output of the decompression 

14 processing module and outputting the stored data to the DAC; and 
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15 a frame rate controller for controlling the compression processing 

16 module to be constant by executing frame interpolating processing, wherein 

17 the decompression processing module has a function of thinning out frames 

18 for continuing reproduction synchronous to voice. 

1 4. The real time recording/reproducing system according to claim 1 , wherein 

2 frame thinning-out in the decompression processing module and the frame 

3 skipping in the decompression processing module are performed preferentially 

4 from frame-interpolation frames to generate digital compressed data involving 

5 much motion. 

1 5. The real time recording/reproducing system according to claim 1, wherein 

2 the compression processing module adds data bit stream data including a 

3 picture header representing a start of a frame compression code, a user data 

4 representing a thinned-out frame and a reference frame code representmg the 

5 same frame as a reference frame. 

1 6. A real time recording/reproducing method for recording a digital data in a 

2 recorder obtained by converting an analog image signal, and reproducing 

3 recorded digital data in an analog data format comprising steps of: 

4 storing the digital data in a first frame memory; 

5 compressing the output of the fhst frame memory; 

6 recording a compressed output of the first frame memory as digital 

7 data in the recorder; 

8 decompressing the digital data read out from the recorder; 

9 storing the decompressed data in a second memory; 

10 controlling the frame rate of the compressed output of the first frame 
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1 1 memory to be constant by executing frame interpolating processing; and 

12 executing frame skipping processing when full frame real time 

13 decompression processing cannot be executed. 

1 7. The real time recording/reproducing system according to claim 6, wherein 

2 the frame thinning-out and the frame skipping operations are performed 

3 preferentially from frame-interpolation frames to generate digital compressed 

4 data involving much motion. 

1 8. The real time recording/reproducing system according to claim 6, wherein 

2 in the compression processing operation data bit stream data including a 

3 picture header representing the start of a frame compression code, a user data 

4 representing a thinned-out frame and a reference frame code representing the 

5 same frame are added as a reference frame. 

1 9. The real time recording/reproducing system according to claim 1, wherein 

2 the compression processing module and the decompression processing 

3 modules are constituted by central processing unit (CPU) software processing 

4 parts. 

1 10. The real time recording/reproducing system according to claim 9, wherein 

2 the ADC is a video capture card and the DAC is a graphic accelerator card. 

1 11. The real time recording/reproducing system according to claim 10, 

2 wherein the first and second frame memories are main memory and video 

3 memory, respectively. 
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1 12. The real time recording/reproducing system according to claim 11, 

2 wherein the recorder is constituted by a hard disc drive. 



Docket 2000P1 20495 (02530022 AA) 
S.N. 09/837,190 



IX. Evidence Appendix 

There is no additional evidence submitted with this Appeal Brief. 
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X. Related Proceedings Appendix 

There are no related proceedings pertaining to this patent application. 

In view of the above, it is requested that the positions of the Examiner be 
reviewed, that the rejections be reversed, and that the application be passed to issue. 
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